Data access and discovery
Contents
Data access and discovery#
Context#
We will be using Long Term Statistics (1999-2019) product provided by the Copernicus Global Land Service over Lombardia and access them through S3-comptabile storage (OpenStack Object Storage “Swift”) and openeo Sentinel-2 Cloud-Optimised Dataset online through STAC.
Data#
This episode we will use data available online or that has been prepared and stored in Zenodo.
Setup#
This episode uses the following Python packages:
xarray
netcdf4
h5netcdf
wget
numpy
s3fs
Please install these packages if not already available in your Python environment. Below, we only install packages that are not available in the EGI-ACE EOSC deployment of Pangeo for the FOSS4G course.
Packages#
In this episode, Python packages are imported when we start to use them. However, for best software practices, we recommend you to install and import all the necessary libraries at the top of your Jupyter notebook.
Introduction to the Long Term statistics#
CGLS LTS are computed over a time span of 20 years aggregated over each 10 days period (month/01,month/11, month/21). For each date the long term minimum, maximum, mean, median and standard deviation are computed.
S3-compatible Object Storage to access online data#
Up to now we downloaded data locally and then open the downloaded file with Xarray open_dataset. Each attendee downloaded a local copy locally. When willing to manipulate large amount of data, this approach is not optimal (requires a lot of unecessary local downloads). Sharing data online as Object Storage allows for data sharing and access to much larger amount of data.
One of the most popular way to access online remote data is through Amazon Simple Storage Service (S3) and you do not necessarily need to use Amazon service to benefit from S3 Object storage. Many other providers offers S3-compatible object storage that can be access in a very similar way.
Below we will be accessing online the NDVI Long Term statistics from Copernicus Land Service that we have publicly stored in OpenStack Object storage (swift).
import s3fs
import xarray as xr
fs = s3fs.S3FileSystem(anon=True,
client_kwargs={
'endpoint_url': 'https://object-store.cloud.muni.cz'
})
Tip
The parameter anon is for anonymous and is set to True because the data we have stored at https://object-store.cloud.muni.cz is public
List files and folders in existing buckets#
Instead of organizing files in various folders, object storage systems store files in a flat organization of containers (called “buckets”).
fs.ls('foss4g-data')
['foss4g-data/CGLS_LTS_1999_2019',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia',
'foss4g-data/test']
fs.ls('foss4g-data/CGLS_LTS_1999_2019_Lombardia')
['foss4g-data/CGLS_LTS_1999_2019_Lombardia/',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0101_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0111_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0121_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0201_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0211_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0221_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0301_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0311_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0321_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0401_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0411_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0421_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0501_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0511_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0521_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0601_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0611_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0621_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0701_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0711_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0721_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0801_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0811_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0821_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0901_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0911_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0921_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1001_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1011_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1021_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1101_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1111_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1121_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1201_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1211_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1221_GLOBE_VGT-PROBAV_V3.0.1.nc']
Access remote files from S3-compatible Object Storage#
s3path = 's3://foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0101_GLOBE_VGT-PROBAV_V3.0.1.nc'
LTS = xr.open_dataset(fs.open(s3path))
LTS
<xarray.Dataset>
Dimensions: (lon: 331, lat: 235)
Coordinates:
* lon (lon) float64 8.5 8.509 8.518 8.527 ... 11.42 11.43 11.44 11.45
* lat (lat) float64 46.7 46.69 46.68 46.67 ... 44.63 44.63 44.62 44.61
Data variables:
crs |S1 b''
min (lat, lon) float32 nan nan -0.076 -0.048 ... 0.216 0.256 0.236
median (lat, lon) float32 nan nan -0.072 -0.048 ... 0.404 0.404 0.4 0.416
max (lat, lon) float32 nan nan -0.028 -0.048 ... 0.56 0.54 0.568 0.588
mean (lat, lon) float32 nan nan -0.064 -0.048 ... 0.388 0.392 0.404 0.42
stdev (lat, lon) float32 nan nan 0.024 nan ... 0.088 0.088 0.088 0.104
nobs (lat, lon) float32 nan nan 4.0 1.0 nan ... 21.0 21.0 21.0 21.0 21.0
Attributes: (12/19)
Conventions: CF-1.6
parent_identifier: urn:cgls:global:ndvi_stats_all
identifier: urn:cgls:global:ndvi_stats_all:NDVI-LTS_1999-2019-0...
long_name: Normalized Difference Vegetation Index
title: Normalized Difference Vegetation Index: Long Term S...
product_version: V3.0.1
... ...
source: Derived from EO satellite imagery
processing_mode: Offline
references: https://land.copernicus.eu/global/products/ndvi
copyright: Copernicus Service information 2021
archive_facility: VITO
history: 2021-03-01 - Processing line NDVI LTSLTS.sel(lat=45.88, lon=8.63, method='nearest')['min'].values
array(0.264, dtype=float32)
Warning
The same dataset can be available from different locations e.g. CGLS distributor VITO, Zenodo, S3-compatible OpenStack Object storage (Swift), etc. How do you know if the analyze the very same dataset? You cannot know except if the datasets have a persistent identifier such as a Digital Object Identifier. It is therefore recommended to be extra careful with where you get your datasets and double check that the content is what you expect (for instance, you can perform basic quality checks).
Access multiple remote files#
s3path = 's3://foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_*.nc'
remote_files = fs.glob(s3path)
remote_files
['foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0101_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0111_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0121_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0201_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0211_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0221_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0301_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0311_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0321_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0401_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0411_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0421_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0501_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0511_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0521_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0601_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0611_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0621_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0701_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0711_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0721_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0801_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0811_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0821_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0901_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0911_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-0921_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1001_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1011_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1021_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1101_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1111_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1121_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1201_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1211_GLOBE_VGT-PROBAV_V3.0.1.nc',
'foss4g-data/CGLS_LTS_1999_2019_Lombardia/AOI_c_gls_NDVI-LTS_1999-2019-1221_GLOBE_VGT-PROBAV_V3.0.1.nc']
We need to add a time dimension to concatenate data. For this, we define a function that will be called for each remote file (via the preprocess parameter of Xarray open_mfdataset.)
from datetime import datetime
def preprocess(ds):
t = datetime.strptime(ds.attrs['identifier'].split(':')[-1].split('_')[1].replace('1999-', ''), "%Y-%m%d")
return(ds.assign_coords(time=t).expand_dims('time'))
Xarray open_mfdataset allows to open multiple files at the same time.
# Iterate through remote_files to create a fileset
fileset = [fs.open(file) for file in remote_files]
When opening remote files, you can also select the variables you wish to analyze.
LTS = xr.open_mfdataset(fileset, combine='nested', concat_dim=['time'], preprocess=preprocess,
decode_coords="all")
LTS
<xarray.Dataset>
Dimensions: (lon: 331, lat: 235, time: 36)
Coordinates:
* lon (lon) float64 8.5 8.509 8.518 8.527 ... 11.42 11.43 11.44 11.45
* lat (lat) float64 46.7 46.69 46.68 46.67 ... 44.63 44.63 44.62 44.61
crs |S1 b''
* time (time) datetime64[ns] 2019-01-01 2019-01-11 ... 2019-12-21
Data variables:
min (time, lat, lon) float32 dask.array<chunksize=(1, 235, 331), meta=np.ndarray>
median (time, lat, lon) float32 dask.array<chunksize=(1, 235, 331), meta=np.ndarray>
max (time, lat, lon) float32 dask.array<chunksize=(1, 235, 331), meta=np.ndarray>
mean (time, lat, lon) float32 dask.array<chunksize=(1, 235, 331), meta=np.ndarray>
stdev (time, lat, lon) float32 dask.array<chunksize=(1, 235, 331), meta=np.ndarray>
nobs (time, lat, lon) float32 dask.array<chunksize=(1, 235, 331), meta=np.ndarray>
Attributes: (12/19)
Conventions: CF-1.6
parent_identifier: urn:cgls:global:ndvi_stats_all
identifier: urn:cgls:global:ndvi_stats_all:NDVI-LTS_1999-2019-0...
long_name: Normalized Difference Vegetation Index
title: Normalized Difference Vegetation Index: Long Term S...
product_version: V3.0.1
... ...
source: Derived from EO satellite imagery
processing_mode: Offline
references: https://land.copernicus.eu/global/products/ndvi
copyright: Copernicus Service information 2021
archive_facility: VITO
history: 2021-03-01 - Processing line NDVI LTSTip
If you use one of xarray’s open methods such as xarray.open_dataset to load netCDF files with the default engine, it is recommended to use decode_coords=”all”. This will load the grid mapping variable into coordinates for compatibility with rioxarray. See rioxarray documentation.
Chunking and compression#
You may have missed it but if you look carefully to LTS, each Data Variable is now a dask.array with a chunsize of (1, 15680, 40320). So basically accessing one data variable would load (1, 15680, 40320) into the memory (and you need quite a large amount of memory to load such dataset!).
For netCDF files, the chunks correspond to the entire size of the dataset. We will see later that other data formats such as zarr are much more optimized for accessing and analyzing large amount of data.
Select Area Of Interest#
Clip data according to a polygon#
We use the same shapefile as the one used in the previous episode.
import geopandas as gpd
try:
GAUL = gpd.read_file('Italy.geojson')
except:
GAUL = gpd.read_file('zip+https://mars.jrc.ec.europa.eu/asap/files/gaul1_asap.zip')
AOI_name = 'Lombardia'
AOI = GAUL[GAUL.name1 == AOI_name]
AOI_poly = AOI.geometry
AOI_poly
14 POLYGON ((10.23973 46.62177, 10.25084 46.61110...
Name: geometry, dtype: geometry
Define the reference system#
LTS = LTS.rio.write_crs(4326)
Once this has been done we can clip data with the polygon that has been read through geopandas at the beginning of the notebook.
LTS = LTS.rio.clip(AOI_poly, crs=4326)
Save Clipped LTS into local zarr#
LTS.to_zarr(rf'c_gls_NDVI-LTS_1999-2019_AOI_VGT-PROBAV_V3.zarr', mode='w')
<xarray.backends.zarr.ZarrStore at 0x7f58520830d0>
Read zarr from local file#
LTS = xr.open_zarr(rf'c_gls_NDVI-LTS_1999-2019_AOI_VGT-PROBAV_V3.zarr')
Visualize LTS with Holoviews#
# visualization
import matplotlib.pyplot as plt
import holoviews as hv
import hvplot.xarray
import hvplot.pandas
LTS['min'][5].hvplot(geo=True, cmap="RdYlGn")
LTS_NDVI_POI = LTS.sel(lat=45.88, lon=8.63, method='nearest')
POI_min = LTS_NDVI_POI['min']
POI_max = LTS_NDVI_POI['max']
POI_std = LTS_NDVI_POI['stdev']
POI_mean = LTS_NDVI_POI['mean']
import numpy as np
(
POI_mean.hvplot(c='grey', label='mean LTS 1999-2019', alpha=0.2, line_dash='dashed') *
POI_min.hvplot(c='b',label='min LTS 1999-2019', alpha=0.2, line_dash='dashed') *
POI_max.hvplot(c='r', label='Max LTS 1999-2019', alpha=0.2, line_dash='dashed') *
hv.Area((POI_mean.time, POI_mean - 2 * POI_std, POI_mean + 2 * POI_std ), vdims=['- Std', '+ Std'], label='Standard deviation LTS 1999-2019',).opts( color='blue', alpha=0.2))\
.opts(title=f"CGLS S3 300m NDVI fluctuation over the year 2022 in comparison with CGLS Long Term Statistics\nPoint of interest Lat {np.round(POI.lat.data,2)} Lon {np.round(POI.lon.data,2)}",
legend_position='bottom_right')
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [27], in <cell line: 2>()
1 (
2 POI_mean.hvplot(c='grey', label='mean LTS 1999-2019', alpha=0.2, line_dash='dashed') *
3 POI_min.hvplot(c='b',label='min LTS 1999-2019', alpha=0.2, line_dash='dashed') *
4 POI_max.hvplot(c='r', label='Max LTS 1999-2019', alpha=0.2, line_dash='dashed') *
5 hv.Area((POI_mean.time, POI_mean - 2 * POI_std, POI_mean + 2 * POI_std ), vdims=['- Std', '+ Std'], label='Standard deviation LTS 1999-2019',).opts( color='blue', alpha=0.2))\
----> 6 .opts(title=f"CGLS S3 300m NDVI fluctuation over the year 2022 in comparison with CGLS Long Term Statistics\nPoint of interest Lat {np.round(POI.lat.data,2)} Lon {np.round(POI.lon.data,2)}",
7 legend_position='bottom_right')
NameError: name 'POI' is not defined
Discover online datasets#
There are so many available online datasets that it can be very difficult to find out where to find the data you need. To ease your work, data catalogs have been defined and allow to browse and search datasets
Spatio Temporal Asset Catalogs (STAC)#
The STAC specification is a common language to describe geospatial information, so it can more easily be worked with, indexed, and discovered.
Openeo#
openEO develops an open API to connect R, Python, JavaScript and other clients to big Earth observation cloud back-ends in a simple and unified way.
Analysis Ready, cloud optimized data (ARCO)#
Accessing LTS from OpenStack Object Storage through a catalog#
The Pangeo forge initiative#
- Online catalogs
- STAC
- Openeo
- Chunking and compression
- Pangeo Forge